For robots to effectively interact with human users, it is necessary that they recognize what people in the environment are doing. This is especially the case when robots are performing complementary tasks since the human users are not following any specific process. There is much uncertainty in how people act and the duration of time they need to perform their actions. In this work, we discuss the use of topic models for such plan and activity recognition tasks. We begin with the development of a domain-independent representation of human postural information obtained from RGB-D sensor data. This representation may be used with Latent Dirichlet Allocation (LDA) topic models as an integration of plan and activity recognition. This is followed by a proposition of extensions to LDA that allow temporal and object relational information to also be used in plan and activity recognition tasks.